Skip to content

fix: ensure raw_stream.response is httpx.Response in langfuse streaming adapter #1409

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix: ensure raw_stream.response is httpx.Response in langfuse streaming adapter #1409

wants to merge 1 commit into from
+12 −10

Conversation

@recursingfeynman
Copy link

@recursingfeynman recursingfeynman commented Oct 17, 2025 •
edited
Loading

When Langfuse is enabled, the object passed to OpenAI’s stream manager as raw_stream is adapter LangfuseResponseGeneratorAsync. The OpenAI manager sets self._response = raw_stream.response and then calls await self._response.aclose(). LangfuseResponseGeneratorAsync previously set response to the AsyncStream itself (not to httpx.Response), so the manager tried to call aclose() on an AsyncStream, causing:

AttributeError: 'AsyncStream' object has no attribute 'aclose'

Steps to reproduce

  • openai == 2.4.0
  • langfuse == 3.7.0
from langfuse.openai import AsyncOpenAI

request_options = {
    "model": "...",
    "messages": [{"role": "user", "content": "Hello"}],
    "n": 1,
}

async with AsyncOpenAI().chat.completions.stream(**request_options) as stream:
    async for event in stream:
        print(event)

Important

Ensure raw_stream.response is httpx.Response by renaming response to stream and setting self.response to stream.response in relevant classes and functions.

  • Behavior:
    • Ensure raw_stream.response is httpx.Response in async adapter by renaming response to stream in _wrap and _wrap_async functions.
    • Set self.response to stream.response in LangfuseResponseGeneratorSync and LangfuseResponseGeneratorAsync classes.
  • Classes:
    • LangfuseResponseGeneratorSync: Renamed response to stream and set self.response to stream.response.
    • LangfuseResponseGeneratorAsync: Renamed response to stream and set self.response to stream.response.
  • Functions:
    • _wrap: Renamed response to stream.
    • _wrap_async: Renamed response to stream.

This description was created by Ellipsis for 816c68a. You can customize this summary. It will automatically update as commits are pushed.

@CLAassistant
Copy link

CLAassistant commented Oct 17, 2025 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Oct 17, 2025

Greptile encountered an error while reviewing this PR. Please reach out to [email protected] for assistance.

@recursingfeynman recursingfeynman changed the title fix: ensure raw_stream.response is httpx.Response in async adapter fix: ensure raw_stream.response is httpx.Response in LangfuseResponseGeneratorAsync Oct 17, 2025
@recursingfeynman recursingfeynman changed the title fix: ensure raw_stream.response is httpx.Response in LangfuseResponseGeneratorAsync fix: ensure raw_stream.response is httpx.Response in langfuse streaming adapter Oct 17, 2025
@recursingfeynman recursingfeynman mentioned this pull request Oct 17, 2025
Open
@hassiebp hassiebp self-requested a review October 20, 2025 11:37
@recursingfeynman
Copy link
Author

recursingfeynman commented Oct 29, 2025

@hassiebp Hi! Maybe add some tests?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hassiebp hassiebp Awaiting requested review from hassiebp

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@recursingfeynman @CLAassistant